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Abstract 


The NASA Dust Management Project (DMP) was established to provide technologies (to TRL 6 development level) 
required to address adverse effects of lunar dust to humans and to exploration systems and equipment, which will 
reduce life cycle cost and risk, and will increase the probability of sustainable and successful lunar missions. The 
technology portfolio of DMP consisted of different categories of technologies whose final product is either a 
technology solution in itself, or one that contributes toward a dust mitigation strategy for a particular application. A 
Decision Gate Process (DGP) was developed to assess and validate the achievement and priority of the dust 
mitigation technologies as the technologies progress through the development cycle. The DGP was part of 
continuous technology assessment and was a critical element of DMP risk management. At the core of the process 
were technology-specific criteria developed to measure the success of each DMP technology in attaining the 
technology readiness levels assigned to each decision gate. The DGP accounts for both categories of technologies 
and qualifies the technology progression from technology development tasks to application areas. The process 
provided opportunities to validate performance, as well as to identify non-performance in time to adjust resources 
and direction. This paper describes the overall philosophy of the DGP and the methodology for implementation for 
DMP, and describes the method for defining the technology evaluation criteria. The process is illustrated by 
example of an application to a specific DMP technology. 

1. Introduction 

The NASA Dust Management Project (DMP) initiated in 2007 1 was established as an element of NASA’s 
Exploration Technology Development Program (ETDP) chartered to address the relevant high priority technology 
needs of multiple elements within the Constellation Program and other ETDP projects. The DMP provided 
technologies (to Technology Readiness Level (TRL) 6 development level) required to mitigate adverse dust effects 
to humans and to exploration systems and equipment, with the anticipated results being reduced life cycle cost and 
risk, and increased probability of sustainable and successful human exploration missions. 


1 DMP was terminated in September 2010 as part of the decision to terminate the Constellation Program. All 
technology development efforts and DGP assessments discussed here date to September 2010. References to 
cancelled and/or restructured programs and projects as they related to the DMP exist in this document for the 
purposes of illustrating former programmatic relationships. At the time of this writing it is unknown which former 
DMP technology development projects have been continued within other programs, but given that they are not 
continued under the auspices of the DMP, descriptions are written in the past tense. 
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The development and implementation of a decision gate process (DGP) to validate the achievement and priority of 
each DMP technology as it progressed through its development cycle was a critical element of DMP risk 
management. The TRLs defined by Mankins [1] for application to NASA-developed technologies provided the basis 
for this process. At the core of the DGP are technology-specific criteria developed to measure the success of each 
technology at attaining the TRLs assigned to each decision gate. In all, four decision gates were delineated with 
TRLs 3 and 4 designated to DMP’s technology development tasks and TRLs 5 and 6 designated to its application 
areas. 

The DMP technology development portfolio consisted of two categories of technologies whose final product was 
either a technology solution in itself (e.g. mechanical components), or one that contributed toward a dust mitigation 
strategy (e.g. electrodynamic dust shield) for a particular application (e.g. surface power systems). The DGP 
accounted for both categories and qualified the technology progression from development starting at TRL 3 through 
to application. The process supported customer milestones by providing opportunities to validate performance as 
well as identify non-performance in time to adjust resources and direction. 

This paper outlines the overall philosophy and methodology of the DGP and identifies the method by which core 
evaluation criteria for each technology category were defined, as well as the definition of specific criteria for each 
technology at a particular TRL. A description of the infrastructure and approach for implementation of the process is 
provided, and is illustrated by an example of application to a DMP technology. 

2. DMP Technologies and Application Areas 

Specific DMP technology development areas included mechanical systems, cleaning technologies, materials 
development, connectors, and thermal and power systems component development. The technology development 
tasks and application areas are listed in Table 1. 

Table 1. DMP Technology Development Tasks and Application Areas 


Technology Development Tasks 

Task 

Scope 

Mechanical 
Components and 
Mechanisms 

Develop mechanical components and mechanisms to include bearings, seals, and gears for 
use in relevant surface assets. The technical approach included performance testing of 
materials and mechanisms with the final product being development of mechanisms and 
novel lubricants. 

Lotus Coatings 

Develop an overcoat coating that has anti-contamination and self-cleaning properties for use 
as a dust mitigation method on various extravehicular activity (EVA), lunar surface system 
(LSS), and Altair surfaces. The unique morphology of the coating was slated to help prevent 
lunar regolith from adhering to coated surfaces and to facilitate dust 
removal/decontamination of those Lotus-coated surfaces. 

SPARCLED 

SPARCLED (Space Plasma Alleviation of Regolith Concentrations in Lunar Environments 
by Discharge) was composed of an electron beam source in the form of an electron gun. A 
flood of electrons rapidly charged any regolith within the beam diameter with a sufficient 
charge to mass ratio to rapidly disperse the dust particles via electrostatic repulsion. 

C0 2 Shower 

To protect entry to an airlock and the habitat, the development of deployable, possibly 
inflatable, dust isolation and removal zones was to be conducted using C0 2 snow shower 
technology. 

Electrodynamic 
Dust Shield 

The electrodynamic dust shield (EDS) used alternating electric fields to dislodge, carry, and 
deposit dust particles off and away from surfaces. The active EDS removed dust particles 
and prevented dust accumulation on surfaces that needed protection, including solar panels, 
in-situ resource utilization (ISRU) optical windows, helmet visors, optical systems, thermal 
radiators, batteries, and boots. 

Mitigation for 

Determine the extent of degradation of thermal control surfaces for Altair and LSS 
components by lunar dust and determine ways to effectively mitigate that degradation. 
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Radiators 


Dust Tolerant EVA- 

Compatible 

Connectors 

Develop connectors (quick disconnects (QD) and umbilical systems) that can be repetitively 
and reliably mated and de-mated during lunar surface EVA. 

Application Areas 

Area 

Scope 

Dust Tolerant EVA 
connectors 

The objectives of this area were component level integration and testing of connectors 
(quick disconnects and umbilical systems) that could be repetitively and reliably mated and 
de-mated during lunar EVAs. These standardized interfaces would be required for structural 
integrity and commodities transfer between linked surface elements. The scope included 
small scale prototype development and component level integration. The components would 
have been integrated into EVA suit and lunar surface system components. 

Dust Mitigation for 
Thermal Control 
Surfaces 

Develop thermal radiator component level technologies to TRL 6 to address adverse lunar 
soil effects. Scaled-up prototypes of thermal control surfaces with the required regolith 
mitigation technologies would be fabricated and tested before infusion. Applicable DMP 
technologies developed to address dust mitigation of thermal control surfaces included the 
EDS, dust repellant materials, Lotus coatings, and other technologies developed by entities 
involved in a separate small business innovative research (SBIR) program. 

Dust Mitigation for 
Surface Power 
Systems 

Dust mitigation technologies for surface power system component level technologies (solar 
panels, energy storage battery modules, interface plates, etc.) were being developed to TRL 
6 in order to address adverse lunar soil effects. The regolith mitigation technologies would 
be integrated into the surface power component subsystem and prototype demonstration 
would be performed in a relevant environment. Applicable DMP technologies developed to 
address dust mitigation of surface power systems include EDS, dust repellant materials, 
Lotus coatings, and other technologies developed by entities involved in the SBIR program. 

Dust Tolerant 

Mechanical 

Components 

Integration and testing of the mechanical component-level technologies identified above in 
the mechanical components and mechanisms technology development task to TRL 6. 

Dust Mitigation for 
ISRU Components 

Integration and testing of ISRU components (optical systems like windows, cameras, light 
sources and sample recovery chambers) with embedded dust mitigation technologies to 
minimize the adverse dust effects during lunar surface operations. Dust mitigation 
technologies would be integrated into the ISRU component subsystem and prototype 
demonstration would be performed in a relevant environment. DMP technologies developed 
to address dust mitigation of ISRU surfaces include EDS, dust repellant materials, Lotus 
coatings, and other technologies developed by entities involved in the SBIR program. 

Dust Mitigation for 
EVA Spacesuit 
Systems 

Integration and testing of EVA components (visors, boots, gloves, TMG over garment, suit 
pressure garment) with embedded dust mitigation technologies to minimize the adverse dust 
effects during lunar surface EVAs. DMP technologies developed to address dust mitigation 
of EVA spacesuit systems include EDS, dust repellant materials, Lotus coatings, and other 
technologies developed by entities involved in the SBIR program. 


3. Need for a Decision Gate Process for DMP Technologies 
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The DGP provided an objective mechanism for overall identification and evaluation of performing and non- 
performing DMP technologies. The process offered frequent opportunities with multiple decision gates to weed out 
non-performing technologies before too many resources were invested in these technologies, and while there was 
time to identify an alternate technology or deviation from an earlier development plan. The DGP also provided an 
opportunity for DMP investments to demonstrate maturity at identified points and on schedule to instill customer 
confidence that each DMP deliverable met qualifying metrics at each step in its progression. The process also 
afforded a mechanism to identify the best performing, highest customer priority technologies, so that during times of 
budget reduction, investments could be allocated appropriately to ensure the most chance of success for those 
technologies. 

3.1 Description of the Decision Gate Process for DMP 


The process of utilizing decision gates is commonly employed in a wide range of applications to assess technology 
developments. The DGP developed specifically for the DMP is described in the following sections. 

3.1.1 Generic Decision Gate Process 

The DGP developed and utilized by the DMP is based on the Technology Stage-Gate™ (TechSG™) described by 
Ajamian and Koen [2] to manage technology development efforts when there is high uncertainty and risk 
surrounding the effort. The TechSG™ spans idea generation and development of a broad understanding of a 
technology that is required, through early technology development until it is decided whether a significant risk has 
been reduced in order for the particular technology to continue its development path. The TechSG™ process utilizes 
a series of gates or reviews where at each gate a certain deliverable or milestone is reviewed to determine whether it 
has been achieved. Deliverables for each subsequent gate may change due to the need to take a different approach 
given unexpected discoveries uncovered in the technology development process. The TechSG™ process brings 
together scientific, financial, and technological elements of a project to ensure that appropriate resources are 
invested into a technology investment according to progress made and knowledge gained, before product 
development may occur. TechSG™ is used to manage a technology development effort until agreed upon risks have 
been satisfied. 

3.1.2 Adaptation to DMP Projects 

The TechSG™ process was adapted for use by the DMP using the same general process elements, but expanded to 
include the review of TRL 3 technologies through to TRL 6 to allow for assessment of technology development 
tasks and application area projects, both of which comprise the DMP portfolio as noted earlier. Hence, the DMP 
DGP has two elements: a) an overall infrastructure similar to the TechSG process [2]; and b) the existing DMP 
technology assessment process and criteria used to evaluate the DMP research and technology development 
portfolio and external technologies of interest since the start of the DMP in 2007 [3]. The DMP DGP implements 
only those aspects of the TechSG™ that support decisions in the development timeframe of technologies for TRL 3- 
6 . 

3.1.3 Purpose of Each Gate and Stage 

The objectives of the DGP are to review each task for its technical maturity, schedule status, ability to meet 
milestones within a given budget, determine the existence and extent of technical hurdles, and to ensure alignment 
with customer priority areas. The TRL definitions were expanded to delineate particular criteria that allow a 
determination of the technologies success to attain a certain TRL. Early in the development of the overall DGP, 
criteria were developed for each technology category (e.g. mechanical components and cleaning systems) to be used 
as a guideline for later definition of specific criteria for individual tasks. 

The elements of the DGP flow include tasks (T), development stages (D), decision gates (G), and infusion points (I) 
as shown in Figure 1. Tasks include individual technology development activities within the DMP that were to be 
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developed to TRL 4, and later integrated with other technologies as part of an overall mitigation strategy for a 
particular application. Development stages are periods of technology development that lead to a decision gate. 
Development stages that lead to decision gates 3 and 4 represent single technology development efforts, while stages 
that lead to decision gates 5 and 6, as notated by an asterisk in Figure 1, represent the development and integration 
of multiple technologies that comprise an application area or mitigation strategy. Efforts occurring in each 
development stage were based on customized expectations of the TRL definitions, particular task objectives, and 
recommendations by the DGP review panel. Infusion points are defined as opportunities for a technology 
application or mitigation strategy to be integrated into a customer’s system. 


Technology Development Application Development 
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Decision Gate Review Process 


T = Task 


D = Development Stage G = Decision Gate I = Infusion Point 

Figure 1. Decision Gate Process Flow 


3.1.4 Decision Gates 

The decision gate element of the process employs the assessment criteria to allow evaluation of a particular 
technology at each of the four TRLs noted previously. All decision gates function in the same way with the end 
result being one of four decisions: Continue, Redirect, Hold, or Terminate as illustrated in Figure 2. 
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Redirect 



Figure 2. Typical Decision Gate with Four Possible Decisions 

A set of decision gate evaluation criteria were developed that reflect the maturity, feasibility, and associated risk 
involved with the ability of a particular technology to reach and pass each decision gate. Evaluation criteria for 
decision gates 5 and 6 also address considerations related to integration, interoperability, and sustainment, which 
become equally important from a systems perspective in an operational environment. Since the technical 
community, DMP, and customer management were represented on the DGP review team, the results lent themselves 
to the basic decisions shown in Figure 2. 

As shown in Figure 3, each DMP decision gate consists of three elements, or fundamental questions. 

• Is this a “good” project? 

The key questions to answer for this gate are: Does this technology satisfy the TRF for this gate? Does it have 
adequate resources and expertise dedicated to its success? This gate is evaluated from strategic, financial, and 
technical points of view. The DGP review team includes the necessary representatives to help evaluate the answer to 
these questions. At that point, a “sub-decision” is made. If the project is good, it continues to the next element. 
Otherwise the project is terminated. 

• Is all information available? 

The key question for this element: Is the task on schedule with achievable technical hurdles? The next sub-gate is to 
determine if the team has completed all the activities and deliverables in a quality fashion. This is important for a 
couple of reasons. First, if the task is not on schedule and activities are not completed, it becomes a risk to the 
completion of the project on time and within budget. Secondly, if the quality of the project deliverables is poor or 
lacking, it may hint at technical or management hurdles that may be insurmountable without redirection. In this 
element, the final decision is to either continue to the next sub-gate, or redirect the project to be evaluated again at 
the earlier element of the decision gate. 

• Technology priority 

In this element, the key question is: What is the impact of the technology and the priority relative to other 
technologies? If the priority is high, the technology is supported and the project is continued. For a low priority 
verdict, there likely is a higher priority project that should be invested instead and the technology is placed on hold 
while another technology is evaluated for inclusion in the DMP portfolio. At this final sub-gate, the final element in 
the DGP is evaluated and completed and the project is given the green light to continue. Should the particular 
project pass all three elements, it would also pass the decision gate being evaluated. 

The evaluation criteria developed for this assessment are prioritized and weighted for given decision gates in order 
to support an objective decision (Figure 3). Decisions that are levied as products of the DGP are final and are to be 
implemented immediately to ensure a timely execution of the required action. Decisions to redirect or to hold the 
project are accompanied by explicit instructions for implementation by DMP Management. 
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Redirect 
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Figure3. Decision Gate Criteria Delineated to Support an Objective Decision 

Decision gates for each DMP technology and application would be coordinated to occur prior to a customer 
milestone for which DMP technology validation was required. Additionally, each gate would be established to 
review a particular technology or application upon the project reaching each TRL 3-6. 

Depending on the results of a particular DGP gate for each technology project reviewed, the same or subsequent 
TRL is reviewed at an appropriate time in the development cycle of the technology or application. 

3.2 Generic Implementation Approach 

The implementation of the DGP for DMP technology development and application areas consists of a sequence of 
steps. The order of these steps is listed below. 

• Collect information on the project using prescribed data sheets 

• Review and score the project by subject matter experts 

• Review of the evaluation by the DGP review team 

• Final decision made by the DGP review team 


3.2.1 Data Collection 

Each project provides detailed information (technical, schedule and cost) in the data sheet (Appendix A). The data 
sheet is reviewed for completeness by DMP Systems Engineering and Integration (SE&I). Missing or supplementary 
information is obtained by follow up contact with the project lead. 

3.2.2. Technical Review and Scoring 

A technical subject matter expert(s) (SME) performs an in-depth review of the information in the data sheet based 
on the assessment criteria in Appendices B or C. The level of fulfillment is determined for each criterion and a score 
is assigned. The weighted scores are summed and then averaged to obtain the final score for the technology or the 
application area. The scoring sheets are provided to the DGP review team. 
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3.2.3 DGP Review Team Evaluation 


The DGP review team consists of the technical subject matter expert and project and customer management. This 
combination of team members allows for objective evaluation of the different technologies, based on scientific and 
technical maturity, feasibility, and relative need for the overall goals of the project. The SME presents the review 
and the scores for the technology at a meeting of the review team. The team evaluates the review and each member 
provides an opinion, based on the assessment criteria, level of fulfillment, and the scores, as well as assessing the 
overall potential of the project, assessing the risks involved and whether the risks can be mitigated, and other 
programmatic factors such as resource constraints (Figure 3). The overall evaluation criteria, which are prioritized 
and weighted, are defined by the DGP review team. The team members also provide recommendations of the 
decision and the rationale for their individual decision regarding the project. The decision will be one of the four 
decisions in Figure 3. 

3.2.4 Final DGP Decision 

The individual team member decisions are compiled as a data package by an executive secretary and are evaluated 
by the entire team. The decisions are counted by voice vote. The final decision (and any related 
conditions/clarifications) is communicated to DMP management, who, in turn, communicates the decision to the 
project lead. 

3.3 Description of Individual Steps in the Process Flow 

The application of the DGP to the DMP technology development and application areas follows the steps described 
above. A stepwise procedure is summarized below (Table 2) that describes an infrastructure for implementing each 
decision gate in terms of required tools, templates, venues, and participants, and defines the format and venue for 
reporting the DGP review team evaluation results to DMP management and to the technology developer. 

Table 2. Individual Steps for Implementation of the DGP for DMP 


Step 

Title 

Description 

Infrastructure 

General Planning Steps 

1 

DGP kickoff 
meeting 

The DGP review team kickoff meeting is by telecon. 
Activities include: nomination of an executive secretary; 
development of an agenda and schedule for the DGP; 
approve the overall DGP plan; approve the list of SMEs. 

Telecon information 

2 

Identify SMEs 

SMEs are proposed in the DGP plan and approved by the 
DGP review team. 

None 

3 

Identify DGP 
review team 
members 

The DGP review team members are proposed and 
approved by DMP Management (MGMT). 

None 
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Step 

Title 

Description 

Infrastructure 

4 

Provide orientation 
to the DGP 

The developers of the DGP plan provide an 
orientation/training session to the DGP review team, 
including instructions on the evaluation process, 
assessment criteria, evaluation forms, and other logistics. 
This session may be included in the kickoff meeting. 

1 . Data sheets 

2. Scoring forms 

3. DGP evaluation 
criteria 

4. DGP evaluation 
forms 

5 

Develop DGP 
schedule 

The detailed DGP schedule for each technology and 
application area is developed by the DGP review team to 
meet the customer milestones. 

None 

6 

Develop decision 
gate evaluation 
criteria 

Overall evaluation pass criteria for each decision gate are 
approved by the DGP review team. These criteria may 
include overall potential, project risks, and other 
programmatic factors (resource constraints). 

None 

Review Steps 

1 

Collect project 
information 

DMP SE&I initiates contact with the project lead to collect 
project information using the data sheet. 

Data sheets 

2 

SME reviews data 
sheet 

The SME performs a review of the data sheet for 
completeness of the information. 

Data sheets 

3 

SME follow up with 
project lead (as 
required) 

The SME follows up with the project lead to obtain any 
missing or supplementary information as required. The 
supplementary information is also recorded in a data sheet. 

None 

4 

SME completes 
review of project 
information and 
scoring 

The SME performs a detailed review of the project 
information based on the assessment criteria and assigns 
individual scores. This will also include the rationale for 
the scores. The weighted score for the project is included 
in the scoring sheet. The SME also includes a summary of 
the review. 

1 . Data sheets 

2. Assessment 
criteria and 
scoring sheets 

5 

Completed SME 
review package 
provided to DGP 
review team 

The completed SME review package (data sheet and 
supplements, scoring sheets, summary) will be provided to 
each DGP Review team member at least 2 weeks prior to 
the team evaluation meeting. 

1 . Review package 

2. Website/Docbase 
for review upload 

6 

SME prepares 
presentation for 
DGP review team 

The SME prepares a presentation of the review for the 
DGP review team. 

Presentation template 

7 

Team convenes 

The DGP review team convenes for the evaluation of the 
project. The SME will present the detailed project review. 
Each team member will provide comments and a summary 

1. Telecon 
information 
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Step 

Title 

Description 

Infrastructure 



of the recommendation for the decision. This will be based 
on the decision gate evaluation criteria. 

2. SME review of 
presentation 
package 

3. Evaluation criteria 

8 

Evaluation data 
package 

Each team member prepares a written summary of their 
recommendation for the decision gate and the rationale for 
the recommendation. The summary and the review data 
package are provided to the DGP review team. 

DGP evaluation data 
package 

9 

DGP review team 
joint evaluation and 
final decision 

The DGP review team meets for a team evaluation of the 
project. The team evaluation is based on team review of 
the individual evaluation data package. The final decision 
is by voice vote of the team members. The final decision is 
recorded in the minutes of the meeting. 

1 . DGP evaluation 
package 

2. Telecon 
information 

10 

Decision 

Communication 

The DGP review team lead communicates the final 
decision to MGMT, who communicates the decision to the 
project lead. 

None 


3.3 Assessment Criteria for DMP Technologies 

The criteria used for the DGP assessment of the DMP technologies are discussed in the following sections. 

3.3.1 TRL Levels 

The TRLs were the metric used to evaluate the maturity of the technologies developed for dust mitigation within the 
DMP. The key observation with regard to the TRL scale is that it only evaluates the maturity of an individual 
technology. TRL takes a given technology from basic principles to concept evaluation through to ‘breadboard’ 
validation, then to prototype demonstration, and finally to completion and successful mission operations. 

For the DMP, the primary end objective of the technology development efforts at the originally planned completion 
of the DMP in 2016 was to successfully demonstrate a dust mitigation technology at TRL 6 that would meet the 
needs of the customer. At the same time for DGP assessments, the technology developments should each have 
matured at least to TRL 3 for typical lunar applications. This is a critical milestone and a decision gate for continued 
DMP support for each technology, since it is unlikely that the technology development projects and application 
areas could fully mature to TRL 6 before the projected completion of DMP in 2016. Therefore, the decision gates 
are assigned at TRL 3, TRL 4, TRL 5 and TRL 6, defined in Table 3 [1]. 
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Table 3. Definitions of NASA TRLs for DMP [1] 


TRL 

Definition 

TRL 3 

Analytical and 
experimental critical 
function and/or 
characteristic proof-of- 
concept 

At this step in the maturation process, active research and development (R&D) 
has been initiated. This must include both analytical studies to set the technology 
into an appropriate context and laboratory-based studies to physically validate 
that the analytical predictions are correct. These studies and experiments should 
constitute “proof-of-concept” validation of the applications/concepts formulated 
at the previous TRL 2. 

Cost to achieve: low ‘unique’ cost (technology specific) 

TRL 4 

Component and/or 
breadboard validation 
in a laboratory 
environment. 

Following successful “proof-of-concepf ’ work, basic technological elements 
must be integrated to establish that the “pieces” will work together to achieve 
concept-enabling levels of performance for a component and/or breadboard. This 
validation must be devised to support the concept that was formulated earlier, and 
should also be consistent with the requirements of potential system applications. 
The validation is relatively “low-fidelity” compared to the eventual system: it 
could be composed of ad hoc discrete components in a laboratory. 

Cost to achieve: low-to-moderate ‘unique’ cost (investment will be technology 
specific, but probably several orders greater than the investment required for 
TRL 3) 

TRL 5 

Component and/or 
breadboard validation 
in a relevant 
environment. 

At this level, the fidelity of the component and/or breadboard being tested has to 
increase significantly. The basic technological elements must be integrated with 
reasonably realistic supporting elements so that the total applications (component 
level, subsystem level, or system level) can be tested in a ‘simulated’ or 
somewhat realistic environment (dust. From one-to-several new technologies 
might be involved in the demonstration. 

Cost to achieve: moderate ‘unique’ cost 

(investment cost will be technology dependent, but is likely to be several orders 
greater than the cost to achieve TRL 4) 

TRL 6 

System/subsystem 
model or prototype 
demonstration in a 
relevant environment 
(ground or space) 

A major step in the level of fidelity of the technology demonstration follows the 
completion of TRL 5. At TRL 6, a representative model or prototype system or 
system - which would go well beyond ad hoc, ‘patch-cord’ or discrete component 
level breadboarding - would be tested in a relevant environment. At this level, if 
the only ‘relevant environment’ is the environment of space, then the 
model/prototype must be demonstrated in space. Of course, the demonstration 
should be successful to represent a true TRL 6. Not all technologies will undergo 
a TRL 6 demonstration: at this point the maturation step is driven more by 
assuring management confidence than by R&D requirements. The demonstration 
might represent an actual system application, or it might only be similar to the 
planned application, but using the same technologies. At this level, several-to- 
many new technologies might be integrated into the demonstration. 

Cost to achieve: technology and demonstration specific 
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While these characterizations are very useful in technology development, they say nothing about how a particular 
technology integrates within a complete system. Even at TRL 6, system-level demonstration is required (Table 3). 
Although the TRL includes no guidance into the uncertainty and risk in moving through the maturation of a TRL, it 
could be correlated to project risk and technological uncertainty for developing a project management framework. 


3.3.2 Technology Needs and Mapping 

The technology prioritization process (TPP) was an activity performed annually by the DMP’s customer, the 
Constellation Program. The TPP provided an opportunity for managers to capture and prioritize their technology 
needs across all projects and mission architectures. TPP results were ordered and ranked by mission architecture and 
priority. Due to the large number of potentially relevant needs related to dust management across the mission 
architectures, a subset of needs was selected by the DMP which represented both the highest ranked needs by the 
TPP and the needs that were closest to the focus of the DMP technology development tasks and the intended project 
scope. The subset of needs for the DMP is presented in Table 4. The DMP technology development tasks are 
mapped to dust-relevant TPP needs in Table 5. 

Table 4. Summary of DMP Assessment Relevant TPP Dust Capability Needs 


TPP# 

Capability Title 

CxP Project 

Mission 

Architecture 

Priority 

344 

Automated lunar regolith cleaning systems 

E&C SIG 

LS 

HD 

601 

Airlock /habitat hatches that are dust sealing 

LSS 

LS 

c 

609 

Dust mitigation of mechanical components 

LSS 

LS 

c 

611 

Dust- tolerant EVA-compatible connectors 

LSS 

LS 

c 

622 

Environmentally robust electrical docking for rover 
communications 

LSS 

LS 

c 

623 

Dust/Regolith mitigation techniques within habitable cabin 

LSS 

LS 

c 

625 

Dust control/remove airborne dust 

LSS 

LS 

c 

627 

Lander dust mitigation 

LL 

LT 

HD 

632 

Long-life high-performance drive train and suspension 
system 

LSS 

LS 

C 

633 

Dust degradation effects and mitigation for thermal control 
system 

LSS 

LS 

c 

E&C SIG = Environments and Controls Special Investigation Group; LSS = Lunar Surface Systems; LL = Lunar Lander; 
LS = Lunar Surface; LT = Lunar Transport; HD = Highly Desirable; C = Critical 
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Table 5. DMP Task Mapping to Assessment Relevant TPP Dust Capability Needs 


DMP Technology Development Task 

TPP Number 

Mechanical Components and Mechanisms 

609, 632 

Lotus Coatings 

601,623,625, 627 

SPARCLED 

623, 625, 627 

C0 2 Shower 

623, 625, 627 

Electrodynamic Dust Shield 

623, 625, 627 

Mitigation for Radiators 

633, 627 

Dust Tolerant EVA-Compatible Connectors 

611,622 


3.3.3 Assessment Criteria 

For assessing the current state of the technologies, a set of criteria were developed that reflect the maturity, 
feasibility, and associated risk involved with the ability of a particular technology to reach each TRL and pass the 
corresponding decision gate, and to eventually meet the DMP dust mitigation performance objectives. Certain 
criteria were ranked and weighted more heavily to reflect the applicability of the technology, and its potential 
impact, as well as the level of research and development effort required to mature the technology to the next TRL up 
to TRL 6. At a minimum, it was expected that the present technologies would have matured to TRL 3. Resource- 
related criteria (cost, schedule, and personnel, technical) were critical in meeting DMP budgets and schedules and 
were accordingly ranked and weighted high. Lower weights were attached to criteria reflecting the ability to 
develop hardware and delivery systems and the associated safety issues for actual application of the technology 
during a lunar mission, since most of the technologies and technology concepts were at very low readiness levels 
(TRL 1-3) for lunar missions. 

Also included were evaluation criteria that addressed considerations relating to integration, interoperability, and 
sustainment which become equally important from a systems perspective in an operational environment. As noted 
earlier, these considerations are not addressed by the TRL. 

Three levels of fulfillment were assigned from high to low for each criterion and a scoring range was provided to 
each fulfillment level. Appendices A and B list the criteria and their individual rank, the scoring range, and the 
weighting factor for the technology development and for application areas, respectively. When a technology has 
matured to TRL 6, the rank and weight for the criteria may be revised to more closely reflect DMP and customer 
requirements for space applications. At that point, mission specific readiness must be evaluated at TRL 7 to TRL 9 
[!]• 

3.3.4. Technology Scoring and Ranking 

For each DMP technology, detailed technical information on proof-of-concept demonstrations and other technical 
achievements, as well as project (cost and schedule performance) information was required in individual project data 
sheets. Each technology data sheet was reviewed in detail by subject matter experts for information related to the 
assessment criteria. In addition, engineering and scientific judgment is applied to the criteria for each technology, as 
well as personal knowledge of the current state-of-the-art for alternative technologies that are used for relevant or 
similar terrestrial applications. In the event the decision is reached to terminate an existing technology development 
project, alternative technologies would be reviewed as possible replacements. The alternative technologies would be 
assessed similarly through the DGP. 
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Based on the reviews, the level of fulfillment is determined for each criterion and a score is assigned from the range 
in Appendices A and B. Once a score is assigned to a criterion, it is multiplied by the weighting factor to obtain the 
weighted score for that criterion. The weighted scores are summed and then averaged to obtain the final score for 
the technology. 


4. Application of the Decision Gate Process to a DMP Technology 

The implementation of the DGP is illustrated by application to the C0 2 snow shower technology. 

4.1 Overview 

The C0 2 shower technology was selected for DGP assessment because it was claimed to be at the point of achieving 
TRL 3. Early implementation of the DGP to this technology at TRL 3 helped determine whether corrections or 
adjustments needed to be made. It also provided the opportunity to verify the implementation DGP and validate its 
application to the DMP technology portfolio. The process was applied as outlined in Section 3.2. 

The C0 2 shower was assessed in December 2009 according to established technical and programmatic criteria that 
were scored and discussed by a review team comprised of MGMT, SE&I, a SME in precision cleaning, and 
Constellation program customer representatives from the LSS, Environmental Control and Life Support Systems 
(ECLSS) and EVA projects. Specific DGP evaluation criteria (Appendix D) were developed for the C0 2 shower 
technology within the general definition of TRL 3 to provide reviewers with a refined meaning of the TRL to allow 
more objectivity in the assessment, and to assist in scoring the technologies. Other criteria included availability of 
resources, technical hurdles to be overcome for successive TRLs, expertise of the task team, and the priority of the 
technology. The DGP evaluation criteria were broader than the technology assessment criteria. 

The review team members scored the DGP evaluation criteria according to three levels of fulfillment with ranges for 
High (67-100), Moderate (34-66) and Low (0-33). The delineation of particular criteria and the associated scoring 
component of the DGP were meant to support the final voice vote of each review team member. Scores were 
calculated per the weighting factor of each criterion. 

4.2 Assessment of the Technology 

The SME presented an overview of the C0 2 shower technology and discussed his assessment, based on the criteria 
in Appendix A. The purpose of this technical assessment was to provide reviewers who were not experts in the 
cleaning of small particles, or the C0 2 shower, with supplemental technical knowledge on the technology itself or a 
category of related technologies from the perspective of an expert in the field. Several questions and issues were 
discussed during the course of the overview of the technology. 

1 . The criteria used by the SME during his technical assessment are organized differently to allow for more 
discussion of the specific technical details of the technology. The review team could choose to use the 
technical expert’s results to supplement their own expertise as they consider how to score the technical 
aspects of the technology. 

2. The C0 2 shower has a critical dependency on successfully integrating with the EDS technology for dust 
particle removal to be useful as an integrated dust mitigation system. Even though no requirement for the 
removal of sub- pm to 10-pm size particles exists for EVA, these two technologies may have been 
synergistic pending test results that identify the smallest particle size that the EDS can remove as a stand- 
alone technology. The C0 2 shower may have effectively extended the lower limit of the EDS applicable 
size range for particle removal, but technology developers would need to know the effective particle size 
range for the EDS to determine how large a crystal would need to grow for the EDS to remove it. Several 
concerns were expressed with the C0 2 shower both as a stand-alone technology and as integrated with the 
EDS: 
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a. The EDS works on a non-charged particle as the EDS self induces a charged state on the particle 
prior to removal. Tests at the time of the review had not determined whether the EDS can induce 
surface charge characteristics when C0 2 crystals have grown onto particles. The task lead 
indicated that the C0 2 may itself need to be charged. 

b. The amount of C0 2 required for the shower to function scales with the volume of the chamber that 
is used for the shower. The task lead had not indicated that C0 2 needs to be recovered or recycled. 
The LSS customer representative for ECLSS noted that the amount of C0 2 required 
(approximately 0.2 m 3 for a shower volume of 2 m 3 ) is not a small amount. 

c. Folds in the outer layers of the EVA suit are a concern as the C0 2 shower is a line-of-sight 
process. It will need to be determined how critical is it to remove dust from all areas of the suit. 

d. There are 2 architectures under consideration that affect potential EVA dust 
mitigation/management technologies. For both architectures, routine switch-out of the suits would 
be the biggest issue for dust mitigation. 

i. SuitPort - Suits would remain outside all the time and would not have to be brought into 
the habitat itself. 

ii. SuitLock - Suits would remain in the airlock and may be bagged if they need to be 
brought into the habitat. 

e. A successful proof-of-concept milestone for the C0 2 shower was achieved by growing crystals on 
a glass slide contaminated with small particles. However, no tests had been performed to show 
that crystals will grow on an EDS, which would be representative of TRL 3. 

f. A set of experiments would have to be designed to show that crystals can grow to a suitable size 
and not be affected by mechanical movement or not collapse under their own weight. 


4.3 Review Team Recommendations 

The final recommendation was given by voice vote. Based on the team member evaluations, the technology scored 
low which would normally support a final voice vote for TERMINATION of the project. However, the technology 
was not yet demonstrated at TRL 3 and the review team rendered a recommendation of REDIRECT with the 
following conditions. 

- REDIRECT 2 only allowed completion of TRL 3 experiments to achieve the definition for this decision gate 
with no additional financial resources. 

- C0 2 shower technology would need to be demonstrated integrated with an EDS. 

- The task lead would need to develop a near-term schedule complete with a test matrix indicating test 
frequency, materials used, and interim task-internal go/no-points. The schedule would not exceed 6 
months, with a preference for completion in less than 3 months. 

- The review team would need to reconvene for a delta decision gate for TRL 3 when the task lead indicates 
he has achieved TRL 3. 

One additional point of consideration is that it would be difficult to evaluate the suitability and applicability of C0 2 
shower technology, and the DMP technologies in general, until it is decided ‘how clean is clean enough’. 

4.4 Lessons Learned 


Over the course of development and implementation of the DGP for use by the DMP, the following points are taken 
as lessons learned for future related activities: 

• Timely application of DGP is essential to assess the technology and make appropriate project adjustments 
(technical, cost and schedule) to meet the overall end objectives. 

• It is important to have consistent evaluation criteria for objective assessment of the technology. 


2 Redirect may have allowed for an orderly closure of the task at no additional cost to the DMP. Data collected 

during Redirect efforts may also justify termination in the future. 
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• The technology must be developed to meet customer needs, rather than conducting a technology 
development project without a well-established application for the technology. 


5. Summary 

The development and implementation of a decision gate process has been described. The DGP was a critical element 
of DMP risk management to validate the achievement and priority of each DMP technology as it progressed through 
its development cycle. At the core of the process are technology-specific criteria developed to measure the success 
of each technology in attaining the TRLs assigned to each decision gate. In all, four decision gates were delineated 
with TRL 3 and 4 assigned and adapted to DMP technology development tasks and TRL 5 and 6 assigned and 
adapted to DMP technology application areas. 

The DMP technology development portfolio consisted of different categories of technologies whose final product 
was either a technology solution itself, or one that contributed toward a dust mitigation strategy for a particular 
application. As described here, the DGP accounts for both categories and qualifies the technology progression from 
development to application. The process supports customer milestones by providing opportunities to validate 
performance as well as identify non-performance in time to adjust resources and direction. 

The overall philosophy and methodology of the DGP for the DMP technologies has been discussed, and the core 
evaluation criteria for each technology category and specific criteria for each technology by TRL were defined. The 
detailed steps in the implementation of the DGP for each decision gate and the supporting infrastructure for 
implementing the process in terms of required tools, templates, venues, and participants have been described. 

Review of the C0 2 snow shower technology has been used to illustrate the implementation of the DGP and its 
application to a DMP technology. The process is applicable to future NASA technology development projects. 
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Appendix A 

DMP DGP Technology Data Sheet 


1. General Information 

• Technology Title: 


• Description: 


• Contact Information (Lead Center, Supporting Centers, External Partners): 


2. Applicability to Dust Needs and Requirements 

Indicate the applicable dust need(s) that this technology or research addresses and indicate the degree of 
potential applicability (Low (L), Moderate (M), High (H)) toward meeting/supporting each need (Select 
needs from the Integrated Dust Needs sheet) 

• Need ID # : Circle or Highlight Low-Mod-High 

• How does the technology or research apply/support? 

• Need ID # : Circle or Highlight Low-Mod-High 

• How does the technology or research apply/support? 

• Need ID # : Circle or Highlight Low-Mod-High 

• How does the technology or research apply/support? 


For Technology Tasks: 

• Describe the potential use scenario and configuration of the technology. 

• Suggest specifications and characteristics of the applied technology to effectively 
mitigate/manage dust for - 

• Lunar application: 

• Martian application: 


3. Developmental / Knowledge Infusion Profile 
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For Technology Tasks: 

• Current NASA TRL: 

• Estimate a reasonable NASA TRL progression profile (calendar year) assuming requested 
funding and optimal development schedule: 

TRL 3: TRL 4: TRL 5: TRL 6: 

• Estimate the probability (Low (L), < 50%; Moderate (M), 50% - 80%; High, >80%) of the 
technology maturing to TRL 6 in the following future years (system PDRs) assuming requested 
funding and optimal development schedule: 

2010: 2011: 2012: 2013: 2014: 

• Describe the major developmental hurdles/issues at each NASA TRL. Indicate decision point 
(go/no-go) for further efforts. 

TRL 3 - 

TRL 4 - 

TRL 5 - 

TRL 6 - 

• List any assumptions or known dependencies. 


4. Supporting Vehicle/Infrastructure Needs and Assumptions: 

For each, discuss anticipated performance improvements, as applicable, for both technology and research 
tasks. 

• Describe significant system or other dependencies, integration issues, or operating assumptions: 

• Estimated Power Demands - 

• Estimated Mass - 

• Estimated Volume - 

• Crew Time (Hours per week) - 
Assume 3 EV As /week 

5. Leveraging Opportunities 
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• Describe current and projected non-NASA applications (or research in applicable areas) that may 
enable or accelerate the development of the technology or research objective independent of 
investment by NASA. 

• Identify Potential Leveraging or Collaborative Activities, and/or Vendors: 

6. Estimate Technology/Research Costs 

• For Technology Tasks: Provide an estimate of development costs to achieve TRL 6. Provide 
cost by TRL and year. 

FY10: TRL 

FY1 1: TRL 

FY12: TRL 

FY13: TRL 

FY14: TRL 

• For Research Tasks: Provide an estimate of costs to meet research objective(s): 

Provide cost by objective and year. 

7. Please provide other relevant information not captured in 1 - 6 above 
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Appendix B 


DMP Technology Development Assessment Criteria 

No. 

Criteria 

Rating 

Rank 

Weighting 

Factor 

High 

Score: 67-100 

Moderate 
Score: 34 - 66 


i 

DMP Needs addressed 

a. Number 

b. Type 

a. Multiple needs addressed or full 
intent of single need addressed 

b. Short-term and long-term 

a. Few needs addressed 

b. Mainly short term 

a. Single need addressed 

b. Mainly short term 

i 

1.0000 


Technology Readiness Level (TRL) 

TRL: 6 and higher 

TRL: 3 to 5 

TRL: below 3 


0.9000 


Technology Impact Level 

a. Technology/research required to 
meet mission/system objectives 

b. Technology/research has major 
impact on meeting mission/system 
objectives. 

Technology/research has 
some impact on meeting 
mission/system objectives. 

Technology/research has a 
minor impact on meeting 
mission/system objectives. 


0.8000 


Development effort to achieve next TRL level 

a. Required or not required 

b. Degree of difficulty 

a. Minimal or no development 

b. Very low degree of difficulty 
anticipated to meet end objectives. 
Probability of success > 99%. 

a. Moderate development 
effort required 

b. High degree of difficulty 
anticipated to meet end 
objectives. Probability of 
success 50 to 98%. 

a. Intensive development 
effort required 

b. Very high degree of 
difficulty anticipated to meet 
end objectives. Probability of 
success < 50%. 


0.7000 


Resources 

a. Cost (personnel, capital, materials, infrastructure, 
travel) 

b. Availability and allocation 

a. Low 

b. High (limited interference with 
other projects) 

a. Moderate 

b. Moderate (some 
interference with other 
projects) 

a. High 

b. Low (significant interference 
with other projects) 

5 

0.6000 


Performance improvement 

a. Improvement (or contributes to improvement) over 
current state-of-the-art (SOA) based on applicable 
figures of merit (FOM), such as reduction in 
percentage of crew time, up mass (weight/volume), 
power requirements 

b. FOM appl 

a. Significant improvement versus 
current SOA 

b. High FOM 

a. Some improvement versus 
current SOA 

b. Moderate FOM 

a. Minor improvement versus 
current SOA 

b. Low FOM 

6 

0.5000 

7 

Technical experience 

a. Team composition, technical background and skills 

a. High 

a. Moderate 

a. Low 

7 

0.4000 

1 

Project Management (PM) 

a. Prior experience of the organization and the 
management and technical team 

b. Controls (budget and schedule, milestones, 
decision gates and deliverable products) 

a. Extensive PM experience 

b. Well defined project controls 

a. Moderate PM experience 

b. Limited project controls 

a. Limited or no formal PM 
experience 

b. Minimal project controls 

■ 

0.3000 

9 

Mission applicability 

a. Types of missions (Transit, Lunar sortie, Lunar 
outpost) 

b. Versatility for a specific mission 

a. Meets multiple missions 

b. Versatility for multiple aspects of a 
single mission or applicable to 
multiple missions 

a. Meets one type of mission 
only 

b. Limited versatility to a 
single aspect of a single 
mission or applicable to single 
mission 

a. Meets one type of mission 
only 

b. No versatility 

9 

0.2000 

10 

Benefit from other similar technology 
development programs 

a. NASA 

b. Non-NASA 

a. Significant leverage off NASA 
programs; or not required 

b. Significant leverage off non-NASA 
programs; or not required 

a. Some leverage off NASA 
programs 

b. Minimal leverage off non- 
NASA programs 

a. Minimal leverage off NASA 
programs 

b. Minimal leverage off non- 
NASA programs 

10 

0.1000 
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Appendix C 


DMP Application Area Assessment Criteria 

No. 

Criteria 

1 Rating 

Rank 

Weighting 

Factor 

High 

Score: 67-100 

Moderate 
Score: 34 - 66 

Low 

Score: 0 - 33 

i 

DMP Needs addressed 

a. Number 

b. Type 

a. Multiple needs addressed or 
full intent of single need 
addressed 

b. Short-term and long-term 

a. Few needs addressed 

b. Mainly short term 

a. Single need addressed 

b. Mainly short term 

i 

1.0000 

2 

Application TRL 

TRL: 6 and higher 

TRL: between 5 and 6 

TRL: 5 or lower 

2 

0.9000 

■ 

Impact Level 

Application has major impact on 
meeting mission/system 
objectives. 

Application has some 
impact on meeting 
mission/system objectives. 

Application has a minor impact 
on meeting mission/system 
objectives. 


0.8000 

4 

Development effort to achieve next TRL 
level 

a. Required or not required 

b. Degree of difficulty 

a. Minimal or no development 

b. Very low degree of difficulty 
anticipated to meet end 
objectives. Probability of 
success > 99%. 

a. Moderate development 
effort required 

b. High degree of difficulty 
anticipated to meet end 
objectives. Probability of 
success 50 to 98%. 

a. Intensive development effort 
required 

b. Very high degree of difficulty 
anticipated to meet end 
objectives. Probability of 
success < 50%. 

4 

0.7000 

5 

Resources 

a. Cost (personnel, capital, materials, 
infrastructure, travel) 

b. Availability and allocation 

a. Low 

b. High (limited interference 
with other projects) 

a. Moderate 

b. Moderate (some 
interference with other 
projects) 

a. High 

b. Low (significant interference 
with other projects) 

5 

0.6000 

6 

Project Management (PM) 

a. Prior experience of the organization and the 
management and technical team 

b. Controls (budget and schedule, milestones, 
decision gates and deliverable products) 

a. Extensive PM experience 

b. Well defined project controls 

a. Moderate PM experience 

b. Limited project controls 

a. Limited or no formal PM 
experience 

b. Minimal project controls 

6 

0.5000 

7 

Integration with other systems - 
infrastructure, utilities, output 
management 

a. Dependency 

b. Degree of difficulty 

a. No multiple interaction 
between systems 

b. Minimal difficulty 

a. Some multiple 
interaction between 
systems 

b. Moderate difficulty in 
integration 

a. Significant multiple 
interaction between systems 

b. High degree of difficulty in 
integration 

7 

0.4000 

8 

Safety, Reliability, Quality and 
Maintainability 

a. Ease of design for low risk and hazard controls 

b. Achieves improvement 

a. High 

a. Moderate 

a. Low 

8 

0.3000 

9 

Technical experience 

a. Team composition, technical background and 
skills 

a. High 

a. Moderate 

a. Low 

9 

0.2000 

10 

Mission applicability 

a. Types of missions (Transit, Lunar sortie, Lunar 
outpost) 

b. Versatility for a specific mission 

a. Meets multiple missions 

b. Versatility for multiple 
aspects of a single mission or 
applicable to multiple missions 

a. Meets one type of 
mission only 

b. Limited versatility to a 
single aspect of a single 
mission or applicable to 
single mission 

a. Meets one type of mission 
only 

b. No versatility 

10 

0.1000 
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Appendix D 


Decision Gate Evaluation Criteria 

No. 

Criteria 

Rating 

Rank 

Weighting 

Factor 

High 

Score: 67-100 

Moderate 
Score: 34 - 66 

Low 

Score: 0 - 33 

Sub-gate 1: Is this a “good” project? 


Technical assessment 

a. SME Review and Score 

b. Meets TRL for this gate 

a. High 

b. Yes 

a. Medium 

b. Yes (conditionally) 

a. Low 

b. No 

1 

0.50 


Resources 

a. Adequacy (personnel, equipment and facilities) 

b. Availability and allocation 

a. High 

b. Well planned and supported 

a. Medium 

b. Medium level support 

a. Low 

b. Limited support 

2 

0.50 

Sub-gate 2: Is all information available? 

1 

Technical hurdles 

a. Number and type 

b. Degree of difficulty to overcome 

a. Low, mostly engineering hurdles 

b. Low 

a. Medium, some basic and some 
engineering hurdles 

b. Moderate 

a. High, many basic and 
engineering hurdles 

b. High 

1 

1.00 


Project Management (PM) 

a. Status (milestones achieved, products delivered) 

b. Controls (budget and schedule) 

a. Large number (> 80%) 

b. Well defined project controls 

a. Medium number (50 to 80%) 
c. Limited project controls 

a. Low number (< 50%) 

b. Minimal project controls 

2 

0.67 


Risk Assessment 

a. Technical 

b. Cost 

c. Schedule 

a. Low 

b. Low 

c. Low 

a. Medium 

b. Medium 

c. Medium 

a. High 

b. High 

c. High 

3 

0.33 

Sub-gate 3: Technology Priority 


1 

Project Impact 

a. DMP/customer needs addressed 

b. Critical support for other DMP technologies 

a. Multiple needs addressed or full 
intent of single need addressed 
(short term and long term) 

b. Yes 

a. Few mainly short-term needs 
addressed 

b. Maybe 

a. Single need (mainly short 
term) addressed 

b. No 

1 

0.50 

2 

Project Priority 

a. Degree of relevance to DMP/customer 

b. Priority relative to other technologies (alternative) 

a. High 

b. High 

a. Medium 

b. Medium 

a. Low 

b. Low 

2 

0.50 
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